Conversation
chenjian2664
force-pushed
the
jack/release-notes-480
branch
2 times, most recently
from
b8633ef to
eb8d35d
Compare
chenjian2664
force-pushed
the
jack/release-notes-480
branch
from
076c0ce to
713a7b2
Compare
chenjian2664
force-pushed
the
jack/release-notes-480
branch
6 times, most recently
from
4c2a302 to
2f7070d
Compare
chenjian2664
force-pushed
the
jack/release-notes-480
branch
4 times, most recently
from
ed7ba17 to
8bf0508
Compare
| `parquet.writer.page-size`, instead. ({issue}`27729`) | ||
| * {{breaking}} Remove the deprecated `gcs.use-access-token` config property. ({issue}`26941`) | ||
| * Optimize Iceberg materialized view freshness checks based on grace period. ({issue}`27608`) | ||
| * Fix failure when reading `$files` metadata table with partition evolution using |
There was a problem hiding this comment.
This one is not clear, either. What does "with partition evolution" mean, concretely? What actions do users have to take to leave a table in that state?
There was a problem hiding this comment.
The user update the partition definition using bucket or truncate on same column, i,e, update partitioning from truncate(a, 10) to truncate(a, 20), without the fix we were encountered:
Invalid schema: multiple fields for name partition.a_trunc
There was a problem hiding this comment.
* Fix failure when reading `$files` metadata tables when a table’s partitioning scheme involving `bucket` or `truncate` changes.
chenjian2664
force-pushed
the
jack/release-notes-480
branch
from
d631d34 to
73bfdb8
Compare
chenjian2664
force-pushed
the
jack/release-notes-480
branch
from
73bfdb8 to
c81862d
Compare
chenjian2664
force-pushed
the
jack/release-notes-480
branch
from
fc2c848 to
2d86427
Compare
|
Another thing for LakeHouse connector release notes, I think it is a good chance to discuss here, current I am going to add the RN of changes that directly on to this connector(which will remove the |
ebyhr
force-pushed
the
jack/release-notes-480
branch
from
acb6ac6 to
b06e5d8
Compare
mosabua
left a comment
mosabua
left a comment
There was a problem hiding this comment.
Only a few minor suggestions left .. very close.
| `truncate` or `bucket` on the same column. ({issue}`26109`) | ||
| * Fix failure when reading `$file_modified_time` metadata column on tables with equality | ||
| deletes. ({issue}`27850`) | ||
| * Avoid Parquet footer explosion when binary columns contain certain pathological values. ({issue}`27903`) |
There was a problem hiding this comment.
is footer explosion a known term .. also "pathological" values sounds like a medical term
There was a problem hiding this comment.
We've already used "Parquet footer" in the past release notes.
@dain @raunaqmorarka Any suggestions for "pathological" keyword?
There was a problem hiding this comment.
Avoid large footers in Parquet files from certain rare string inputs.
ebyhr
force-pushed
the
jack/release-notes-480
branch
from
b06e5d8 to
6af29f1
Compare
|
|
||
| ## General | ||
|
|
||
| * Add coordinator and worker counts to the metrics endpoint. ({issue}`27408`) |
There was a problem hiding this comment.
"metrics endpoint" is too vague. Specify which API is affected.
| ## General | ||
|
|
||
| * Add coordinator and worker counts to the metrics endpoint. ({issue}`27408`) | ||
| * Add `task.scale-writers.max-writer-memory-percentage` configuration property |
There was a problem hiding this comment.
How does this improve memory management? I can't tell from the release note entry or from the linked PR description.
There was a problem hiding this comment.
I think the PR just made it configurable, and the "improve" doesn't mean improving memory efficiency.
@sopel39
There was a problem hiding this comment.
It's just easier to manage peak memory usage by writer tasks using this property
There was a problem hiding this comment.
In that case, I'd suggest:
* Allow configuring the maximum amount of memory to use while writing tables through the xxx configuration property.
| configuration properties. ({issue}`27637`) | ||
| * Improve performance for remote data exchanges on newer CPU architectures and Graviton | ||
| 4 CPUs. ({issue}`27586`) | ||
| * Improve performance of queries with data exchanges or aggregations. ({issue}`27657`) |
| * Improve performance for remote data exchanges on newer CPU architectures and Graviton | ||
| 4 CPUs. ({issue}`27586`) | ||
| * Improve performance of queries with data exchanges or aggregations. ({issue}`27657`) | ||
| * Reduce out-of-memory errors in window queries when spilling is enabled. ({issue}`27873`) |
There was a problem hiding this comment.
There's no such a thing as a "window query". Rephrase to "... for queries involving window functions..."
| * Reduce out-of-memory errors in window queries when spilling is enabled. ({issue}`27873`) | ||
| * Fix incorrect results when using {func}`localtimestamp(3)`. ({issue}`27806`) | ||
| * Fix {func}`localtimestamp` failure for precisions 7 and 8. ({issue}`27807`) | ||
| * Fix spurious query failures when querying `system` catalog while some catalog |
There was a problem hiding this comment.
Fix spurious query failures when querying the `system` catalog during catalog drop operations.
| corresponding catalog session property `extended_statistics_enabled` are now defunct. ({issue}`27914`) | ||
| * Improve effectiveness of bloom filters for high cardinality columns written in Parquet files. ({issue}`27656`) | ||
| * Improve query performance when querying fresh Iceberg materialized view. ({issue}`27608`) | ||
| * Reduce planning time of queries with scans on tables with delete files. ({issue}`27955`) |
There was a problem hiding this comment.
... of queries on tables containing delete files
| * {{breaking}} Remove the `hive.fs.new-file-inherit-ownership` configuration property. ({issue}`28029`) | ||
| * {{breaking}} The catalog configuration property `iceberg.extended-statistics.enabled` and the | ||
| corresponding catalog session property `extended_statistics_enabled` are now defunct. ({issue}`27914`) | ||
| * Improve effectiveness of bloom filters for high cardinality columns written in Parquet files. ({issue}`27656`) |
| * {{breaking}} The catalog configuration property `iceberg.extended-statistics.enabled` and the | ||
| corresponding catalog session property `extended_statistics_enabled` are now defunct. ({issue}`27914`) | ||
| * Improve effectiveness of bloom filters for high cardinality columns written in Parquet files. ({issue}`27656`) | ||
| * Improve query performance when querying fresh Iceberg materialized view. ({issue}`27608`) |
There was a problem hiding this comment.
... when querying a fresh materialized view
| `parquet.writer.page-size`, instead. ({issue}`27729`) | ||
| * {{breaking}} Remove the deprecated `gcs.use-access-token` configuration property. ({issue}`26941`) | ||
| * {{breaking}} Remove the `hive.fs.new-file-inherit-ownership` configuration property. ({issue}`28029`) | ||
| * {{breaking}} The catalog configuration property `iceberg.extended-statistics.enabled` and the |
There was a problem hiding this comment.
"Remove support for the xxx configuration option and yyy session property."
|
|
||
| ## Oracle connector | ||
|
|
||
| * Add support for configuring a connection wait timeout with the `oracle.connection-pool.wait-timeout` |
There was a problem hiding this comment.
Add support for configuring the connection wait timeout through the `oracle.connection-pool.wait-timeout` catalog property.
0e739e7 to
199d906
Compare
ebyhr
force-pushed
the
jack/release-notes-480
branch
2 times, most recently
from
edc9980 to
b12cab8
Compare
ebyhr
force-pushed
the
jack/release-notes-480
branch
from
b12cab8 to
1b6482e
Compare
|
Updated. |
|
This PR has been updated as of Feb 12. |
|
This PR has been updated as of Feb 15. |
ebyhr
force-pushed
the
jack/release-notes-480
branch
from
1b6482e to
65d0306
Compare
|
This PR has been updated as of Feb 16. |
| * Allow configuring the maximum amount of memory to use while writing tables | ||
| through the `task.scale-writers.max-writer-memory-percentage` configuration | ||
| property. ({issue}`27874`) | ||
| * Add `array_first(array(E), function(E, boolean))` to return an element |
There was a problem hiding this comment.
* Add variant of {func}`array_first` for finding the first element that matches a predicate.
There was a problem hiding this comment.
The {func}`array_first` generates a link to array_first(array(E)) which doesn't have the second parameter. Is it okay?
| * Fix {func}`localtimestamp` failure for precisions 7 and 8. ({issue}`27807`) | ||
| * Fix spurious query failures when querying the `system` catalog during catalog | ||
| drop operations. ({issue}`28017`) | ||
| * Fix failure when executing `date_add` function with larger value. ({issue}`27899`) |
There was a problem hiding this comment.
A value greater than Integer.MAX_VALUE.
| * Sort stages in the query details page numerically rather than alphabetically. ({issue}`27655`) | ||
|
|
||
| ## CLI | ||
| * Fix warning generated when system terminal cannot be created. ({issue}`28321`) |
There was a problem hiding this comment.
This doesn't need to be mentioned.
| writing to Delta Lake tables on S3. ({issue}`27701`) | ||
| * Fix potential table corruption when executing `CREATE OR REPLACE` with table definition | ||
| changes. ({issue}`27805`) | ||
| * Fix Azure Storage connectivity issues. ({issue}`28058`) |
There was a problem hiding this comment.
What kind of connectivity issues?
| * {{breaking}} Remove the `hive.fs.new-file-inherit-ownership` configuration property. ({issue}`28029`) | ||
| * Improve the effectiveness of Bloom filters for high-cardinality columns in Parquet files. ({issue}`27656`) | ||
| * Fix Azure Storage connectivity issues. ({issue}`28058`) | ||
| * Fix memory accounting for insert queries on a bucketed sorted table. ({issue}`28315`) |
There was a problem hiding this comment.
* Fix incorrect memory accounting for `INSERT` queries targeting bucketed and sorted tables.
| * Add support for column default values in Iceberg v3 tables. ({issue}`27837`) | ||
| * Add support for creating, writing to or deleting from Iceberg v3 tables. ({issue}`27786`, {issue}`27788`) | ||
| * Add `content` column to `$manifests` and `$all_manifests` metadata tables. ({issue}`27975`) | ||
| * Add support for changing nested types in `array` and `map` types. ({issue}`27998`) |
There was a problem hiding this comment.
* Add support for changing `map` and `array` nested types through `ALTER ... SET DATA TYPE`.
| `extended_statistics_enabled` session property. ({issue}`27914`) | ||
| * Improve the effectiveness of Bloom filters for high-cardinality columns in Parquet files. ({issue}`27656`) | ||
| * Improve query performance when querying a fresh materialized view. ({issue}`27608`) | ||
| * Improve `optimize_manifests` to cluster manifests by partitions for improving |
There was a problem hiding this comment.
* Enhance `optimize_manifests` to cluster manifests by partition, improving read performance for queries that apply partition filters.
ebyhr
force-pushed
the
jack/release-notes-480
branch
from
65d0306 to
e1a2570
Compare
|
Addressed comments except for #27719 (comment) |
8 participants
Description
Release notes for 480
Additional context and related issues
Release notes
Format: PR/issue number, ✅ / ❌ rn
✅ rn - release note added and verified, or assessed to be not necessary, set to ❌ rn before completion
15 Dec 2025
16 Dec 2025
npm clean installintrino-web-uiconfig #27625 ✅17 Dec 2025
18 Dec 2025
19 Dec 2025
20 Dec 2025
21 Dec 2025
ImmutableList.copyOfinJdbcTableHandleconstructor #27710 ✅22 Dec 2025
$filestable in Lakehouse connector #26830 ❌23 Dec 2025
delete_after_commit_enabledandmax_previous_versionsIceberg table properties #27717 ✅24 Dec 2025
25 Dec 2025
26 Dec 2025
PlanNodecomment #27761 ✅DefaultJdbcMetadata#27762 ✅27 Dec 2025
29 Dec 2025
30 Dec 2025
deprecated.join-pushdown.with-expressionsrelease entry #27718 ✅31 Dec 2025
01 Jan 2026
02 Jan 2026
WHEN STALEin analyzer #27502 ❌whenStaleBehaviornon-optional inMaterializedViewDefinition#27821 ✅WHEN STALEclause #27800 ✅04 Jan 2026
05 Jan 2026
06 Jan 2026
table_changesin Iceberg connector #27847 ✅07 Jan 2026
08 Jan 2026
gcs.use-access-tokenfrom gcs filesystem #26941 ❌fileSysteminstead create new one #27885 ✅09 Jan 2026
TestDeltaLakeLocalConcurrentWritesTest.testConcurrentInsertsSelectingFromTheSameTemporalVersionedTable#27855 ✅10 Jan 2026
11 Jan 2026
12 Jan 2026
IcebergQueryRunner#27894 ✅newTrinoTableinTestIcebergV2#27888 ✅13 Jan 2026
14 Jan 2026
15 Jan 2026
16 Jan 2026
CREATE OR REPLACEwith different schema #27886 ✅19 Jan 2026
DeleteVector#27938 ✅20 Jan 2026
FLOATtype inQuerytable function in Oracle #27880 ✅TestIcebergBigLakeMetastoreConnectorSmokeTest#testCreateTableWithTrailingSpaceInLocationfailure #27951 ✅returnwithcontinuein SET DATA TYPE tests #27958 ✅21 Jan 2026
validateTableForTrino#27972 ✅22 Jan 2026
23 Jan 2026
24 Jan 2026
25 Jan 2026
26 Jan 2026
27 Jan 2026
28 Jan 2026
hive.fs.new-file-inherit-ownershipconfig #28029 ✅29 Jan 2026
30 Jan 2026
S3PresignerfromS3FileSystemLoader#28038 ✅02 Feb 2026
03 Feb 2026
file.encoding=UTF-8#28084 ✅04 Feb 2026
05 Feb 2026
date_addfunction with larger value #28083 ✅06 Feb 2026
07 Feb 2026
08 Feb 2026 ✅
09 Feb 2026 ✅
10 Feb 2026 ✅
11 Feb 2026 ✅
12 Feb 2026 ✅
13 Feb 2026 ✅
15 Feb 2026 ✅
16 Feb 2026 ✅
number, high precision decimal type #28120TransactionLogAccess#2830717 Feb 2026 ✅
ResultSetMetaData.getColumnClassName#2831418 Feb 2026 ✅
19 Feb 2026 ✅
20 Feb 2026 ✅
#/?chars in Google Cloud Storage paths #2829221 Feb 2026 ✅
22 Feb 2026 ✅
23 Feb 2026 ✅
24 Feb 2026
25 Feb 2026
26 Feb 2026
27 Feb 2026
28 Feb 2026
01 Mar 2026
02 Mar 2026
03 Mar 2026